Data Validation Readme.txt

This sample illustrates how the composite datasource can be used to construct a data validation process.
This sample uses the derby database to stage the processed data throughout the validation process.
Auditing reports are being generated at the end of each validation process.

Requirements
---------------------
The Derby Database Server is required for this sample.

Database Preparation
--------------------

1/ Unzip file "repository:/ElixirSamples/DataSource/Data Validation/zip/Customer.zip" to your derby folder
   ( e.g. C:\Derby\bin\Customer )
2/ Modify "repository:/ElixirSamples/DataSource/Data Validation/connection pool/Customer.pool" URL field to 
   reference the Derby database, (eg. "jdbc:derby://localhost:1527/Customer") 
3/ Start the Derby Server, ensure that the connection pool is active by testing the connection via the "Test Connection" button

ETL
---
- The file "repository:/ElixirSamples/DataSource/Data Validation/Main.ds" is the main ETL process. 
- This ETL process validates the "State_Province" field to ensure that only valid 2 letter codes are 
  certified "pass" while others "fail".
- DataDrop_001 and DataDrop_002 are used to stage the data in the Derby Database for inspection as 
  it passes through the validation process.

Job
---
- The file "repository:/ElixirSamples/DataSource/Data Validation/jobs/Main.job" executes the following
  - Start the ETL process
  - Generate the audit reports
  - If an error occurs, an email will be sent.
- This job file can be scheduled to be executed on a regular basis

Maintenance
------------

- The file "repository:/ElixirSamples/DataSource/Data Validation/Maintenance.ds"
  can be executed to reset the sample by dropping all records from the DataDrop Tables.
